A variational characterization of Rényi divergences

نویسنده

  • Venkat Anantharam
چکیده

Atar, Chowdhary and Dupuis have recently exhibited a variational formula for exponential integrals of bounded measurable functions in terms of Rényi divergences. We develop a variational characterization of the Rényi divergences between two probability distributions on a measurable space in terms of relative entropies. When combined with the elementary variational formula for exponential integrals of bounded measurable functions in terms of relative entropy, this yields the variational formula of Atar, Chowdhary and Dupuis as a corollary. We also develop an analogous variational characterization of the Rényi divergence rates between two stationary finite state Markov chains in terms of relative entropy rates. When combined with Varadhan’s variational characterization of the spectral radius of square matrices with nonnegative entries in terms of relative entropy, this yields an analog of the variational formula of Atar, Chowdary and Dupuis in the framework of finite state Markov chains.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Rényi Divergence Variational Inference

This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi’s α-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is controlled by the value of α that parametrises the divergence. The reparameterizati...

متن کامل

Variational Inference with Rényi Divergence

This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi’s α-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is controlled by the value of α that parametrises the divergence. The reparameterizati...

متن کامل

Different quantum f -divergences and the reversibility of quantum operations

The concept of classical f -divergences gives a unified framework to construct and study measures of dissimilarity of probability distributions; special cases include the relative entropy and the Rényi divergences. Various quantum versions of this concept, and more narrowly, the concept of Rényi divergences, have been introduced in the literature with applications in quantum information theory;...

متن کامل

Reversibility of stochastic maps via quantum divergences

The concept of classical f -divergences gives a unified framework to construct and study measures of dissimilarity of probability distributions; special cases include the relative entropy and the Rényi divergences. Various quantum versions of this concept, and more narrowly, the concept of Rényi divergences, have been introduced in the literature with applications in quantum information theory;...

متن کامل

On Rényi measures and hypothesis testing

We provide a variational characterization for the various Rényi information measures via their Shannon counterparts, and demonstrate how properties of the former can be recovered from first principle via the associated properties of the latter. Motivated by this characterization, we give a new operational interpretation for the Rényi divergence in a twosensor composite hypothesis testing framew...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2017